Repository navigation
fix(agent): follow a parallel Stream's map, filter, reduce and collect lambdas (#1172) - #1209
Merged
Merged
Conversation
…t lambdas (#1172) Only forEach's action carried the run to the common pool's workers (#1156), so a race inside a parallel map lambda was seen on the caller's own elements alone. A lambda given to Stream.map, filter or peek now waits in a slot. The woven terminal operation (forEach, forEachOrdered, collect, reduce, min, max, the matches, toList, toArray, count, the finds) lends the run to every slot of its pipeline while it runs, and its own lambdas run like forEach's action. A pipeline ended by an unwoven terminal such as iterator() lends nothing: with no join to order the caller after the helpers, lending would report the read of a result as a race. A combiner, a reduce operator or a comparator merges what other subtasks built, after the JDK's fork and join, which the agent does not see. Each element now publishes its thread's clock, the caller's included, and those three receive every clock published so far before they run, so a correct three-argument collect stays silent. Not followed yet: a primitive stream's map and filter, a stage behind an unwoven one (sorted, distinct, limit, mapToInt) and a Collector's own functions; documented in the limitations. Tests first: the pipeline and terminal attribution tests, then the two combiner-ordering tests, which reported a correct collect and reduce as races until the combine rule. Corpus: four new idioms, the two broken ones fire PROMPT/HIGH and the two correct ones stay silent, 0 false positives, every gate green. Library suite 3921 green, agent suite green. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Codecov Report❌ Patch coverage is 📢 Thoughts on this report? Let us know! |
…1172) Corpus Eval went red on JDK 25 and 26: SharedCollectionDetector reported the correct three-argument collect (idiom_parallelCollect_containerPerSubtask) as concurrent ArrayList writes. The corpus JVM had started about 18,000 workers by then, and a happens-before clock keeps 4,096 threads (#840), dropping the lowest ids first. The common pool's threads are started early and keep working, so their ids are below every ended worker's, and they were the first entries dropped. A combine on one pool thread received the clock of the pool thread that filled the container it merges and dropped that entry in the same join, so the merge looked unordered. The same loss hit the caller's read after a parallel forEach (#1156) once a suite had met that many threads. Past the cap, entries of threads that have ended now go first, lowest ids first, then the lowest ids of running ones. Any choice still only loses an edge, never adds one. The model keeps a weak id-to-thread map so the over-cap join can ask whether a thread is still running. Verified: the new AgentParallelStreamHooksTest case (a caller that met 4,104 threads, then a correct collect) failed on JDK 26 before the fix, and its full clock also turned three existing ordering tests red in the same JVM; HappensBeforeTest.theCapDropsEndedThreadsFirst failed before the fix. After it: library suite 3923 green on JDK 26, static gates green on JDK 21, corpus eval on JDK 26 green in every lane (idiom lane 82/82). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Test Suite (21) failed in DetectorAnalysisUnderConcurrentWritesTest: CompletableFutureObtrudeDetector.analyze() threw a NullPointerException while another thread was still recording. recordObtrude creates the future's state before it sets lastObtrudedByThread, and analyze() passed that null to Map.of, which threw. ifIssue then dropped the run's whole report. analyze() now reads the field once and skips a state whose thread is not set yet. The analysis at the end of the run reports it. Verified on JDK 21: the new anObtrudeStillBeingRecordedDoesNotBreakAnalysis failed before the fix and passes after it, with DetectorAnalysisUnderConcurrentWritesTest green; static gates green. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
) Corpus Eval stayed red on JDK 25 and 26: SharedCollectionDetector still reported the correct three-argument collect (idiom_parallelCollect_containerPerSubtask) after the clock-cap fix. From JDK 24 the common pool's workers are innocuous: each time one goes idle and wakes up again it clears its thread-locals, often partway through one parallel stream. The thread's clock lives in the CLOCKS thread-local, so the next access started a fresh clock, whose own count began again and which had forgotten everything the thread acquired. A combine on that thread then merged a container it had filled itself and read the merge as unordered. A CI trace showed it: worker-3's own count was 380 when it wrote the list, while the clocks it published carried 126 for worker-1, whose clock was at 252. JDK 21 does not clear them, which is why it never failed. start() now returns the clock THREADS already holds for the thread, so the same thread keeps the same clock. Verified on JDK 25: the new aClearedThreadLocalKeepsTheClock failed before the fix and passes after it; the full library build with all gates (3925 + 113 + 2 tests) and corpus-eval (139 tests) are green. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ambdas # Conflicts: # docs/CHANGELOG.md
This was referenced Nov 7, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Partly addresses #1172 (reference
Streampipelines; primitive streams' intermediate operations stay open).Why
Only a parallel
forEach's action carried the run to the common pool's workers (#1156). A plain counter or a sharedArrayListinside a parallelmaplambda was seen only on the caller's own elements, so the commonest parallel-stream mistakes afterforEachdrew nothing reliable.A lambda given to
Stream.map,filterorpeeknow waits in a slot. The woven terminal operation (forEach,forEachOrdered,collect,reduce,min,max, the matches,toList,toArray,count, the finds) lends the run to every slot of its pipeline while it runs, and runs its own lambdas likeforEach's action. A pipeline ended by an unwoven terminal such asiterator()lends nothing. With no join there to order the caller after the helpers, lending would report reading a result as a race.A combiner, a
reduceoperator or a comparator merges what other subtasks built, after a fork and join the agent does not see. Each element now publishes its thread's clock, the caller's included, and those three receive every clock published so far before they run.How it was verified
AgentParallelStreamHooksTest: pipeline and terminal lambdas run for the run on helpers, the caller is ordered after a map lambda oncecollectreturns, aniterator()pipeline lends nothing, and every woven form returns the stream's own answer.collectand areduceover objects built on other workers as races. Both now pass, and the class passed three consecutive runs.parallelMap_plainCounterandparallelMap_addsToAnArrayListfirePROMPT/HIGH;parallelMap_countsUnderALockandparallelCollect_containerPerSubtaskstay silent. 0 false positives, every gate green.SubstitutionHooksArePointsTest.Checklist
VIBETAGS-START/VIBETAGS-ENDmarkersAgentConcurrencyUtilHooks, additive, tagged@since 1.14.2. Change the tag if the next release gets another number.Co-Authored-BytrailerAnything reviewers should push back on
WeakHashMapkeyed by the stream, only inside a run.mapandfilter, a stage behind an unwoven one (sorted,distinct,limit,mapToInt), and aCollector's own functions. All of these run unattributed as before and are documented indocs/agent/limitations.md.Second commit: the clock cap dropped the common pool's threads
Corpus Eval failed on JDK 25 and 26:
idiom_parallelCollect_containerPerSubtask, a correct three-argumentcollect, drew a SharedCollectionDetector finding. By then the corpus JVM had started about 18,000 workers. A happens-before clock keeps 4,096 threads (#840) and dropped the lowest ids first. The common pool's threads have the lowest ids and are still running, so a combine dropped the entry of the pool thread whose container it was merging. The same loss affected the read after a parallelforEach(#1156) in any suite that had started that many threads.HappensBefore.Stamp.joinnow drops ended threads first, lowest ids first, then the lowest ids of running ones. Either way it only ever loses an edge.AgentParallelStreamHooksTest.aCombinerStaysOrderedPastTheClockCapandHappensBeforeTest.theCapDropsEndedThreadsFirstboth failed on JDK 26 before the fix. The first one's full clock also turned three existing ordering tests red in the same JVM.HappensBeforeclass andjoinjavadoc. No VibeTags guardrail added: the drop rule is documented where it lives, and no external contract changed.Third commit: obtrude analysis vs. a record still being written
Test Suite (21) failed in
DetectorAnalysisUnderConcurrentWritesTest:CompletableFutureObtrudeDetector.analyze()threw a NullPointerException while another thread was still recording, because the thread name was not set yet and the null went intoMap.of. This bug was already there before this PR (the test came with #1150).analyze()now skips a record that is not complete yet. The new test failed without the fix and passes with it, on JDK 21.🤖 Generated with Claude Code